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IMAGE PROCESSING METHOD AND APPARATUS 

BACKGROUND OF THE INVENTION 
Field of the Invention 
5 This invention relates to an image processing method 

and apparatus for laying out person images, which have been 
obtained by photographing a plurality of persons, and forming 
Q a layout image, such as a school photo. This invention also 

^ relates to a recording medium, on which a program for causing 

^10 a computer to execute the image processing method has been 

y recorded and from which the computer is capable of reading the 

program. 

^ Description of the Prior Art 

15 Album prints , typically school photos , are formed 

15 by laying out person images, which have been obtained by 

photographing persons, in predetermined layouts, thus 
obtaining layout images, and printing the layout images. As 
illustrated in Figure 5, if the person images are merely laid 
out in a layout image comprising the person images, positions 
20 and/or sizes of face patterns of the persons will vary for 

different person images, and the layout image will become 
unsightly. Therefore, it may be considered to take photographs 
such that the positions and/or the sizes of the face patterns 
of the persons in the photographs may be uniform. However, in 
25 cases where photographs of a plurality of persons are taken, 
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considerable time and labor are required to take the photographs 
by changing the position of the camera in accordance with the 
heights of the persons and/or the sizes of the faces of the 
persons. Also, the position of the camera with respect to the 
persons varies for different cameramen and different dates of 
photographing. Accordingly, it is difficult to take the 
photographs such that the positions and/or the sizes of the 
face patterns of the persons in all of the photographs may be 
uniform. 

Therefore, in cases where a layout image is to be 
formed, when operations for printing from sheets of negative 
film, on which person images have been recorded, are performed, 
the positions of the sheets of negative film and/or the 
positions of photosensitive materials have heretofore been 
adjusted such that the positions and/or the sizes of the face 
patterns of the persons may be uniform among the images. 
Alternatively, a layout image has heretofore been formed by 
laying out person images , which have been obtained by performing 
a trimming process such that the positions and/or the sizes 
of the face patterns of the persons may be uniform. As another 
alternative, a layout image has heretofore been formed by 
printing person images to a comparatively large size, arraying 
the prints of the person images such that the positions of the 
face patterns of the persons may be approximately uniform, and 
again performing the photographing operation. 



However, with the technique for forming a layout 
image by adjusting the positions of the sheets of negative film 
and/or the positions of the photosensitive materials and 
performing the printing operations, and with the technique for 
forming a layout image by arraying the printed images again 
performing the photographing operation, the efficiency of the 
layout image formation and the productivity of the layout images 
cannot be kept high. 

SUMMARY OF THE INVENTION 

The primary object of the present invention is to 
provide an image processing method, wherein a layout image, 
such as a school photo, is capable of being formed efficiently. 

Another object of the present invention is to provide 
an apparatus for carrying out the image processing method. 

The specific object of the present invention is to 
provide a recording medium, on which a program for causing a 
computer to execute the image processing method has been 
recorded and from which the computer is capable of reading the 
program. 

The present invention provides an image processing 
method for obtaining a layout image signal representing a layout 
image, in which a plurality of person images are laid out, from 
a plurality of original image signals, each of the original 
image signals representing a person image, in which a face 
pattern of a person is embedded, the method comprising the steps 



i) detecting face information from each of the 
original image signals, the face information representing a 
position and/or a size of the face pattern of the person in 
the person image represented by each original image signal, 

ii) performing a face pattern normalizing process 
on each of the original image signals and in accordance with 
the detected face information, a plurality of normalized image 
signals being obtained from the face pattern normalizing 
process , and 

iii) laying out a plurality of images, which are 
represented by the normalized image signals, in a predetermined 
layout, whereby the layout image signal representing the thus 
formed layout image is obtained. 

The term "face information" as used herein means, 
for example, the information representing a displacement 
quantity for transformation of a person image such that the 
position of the face pattern in the person image may coincide 
with a position in a predetermined reference image or a 
predetermined coordinate position. Also, the term "face 
pattern normalizing process" as used herein means a process 
for transforming the original image signal with an affine 
transformation technique, or the like, such that the positions 
and/or the sizes of the face patterns of the persons may become 
uniform among the person images . 



The present invention also provides an image 
processing apparatus for obtaining a layout image signal 
representing a layout image, in which a plurality of person 
images are laid out, from a plurality of original image signals, 
5 each of the original image signals representing a person image, 

in which a face pattern of a person is embedded, the apparatus 
comprising: 

0 i) detection means for detecting face information 
=^ from each of the original image signals, the face information 

%J 10 representing a position and/or a size of the face pattern of 

1 y 

^ the person in the person image represented by each original 

image signal, 

==5 ii) normalization means for performing a face 

Q pattern normalizing process on each of the original image 

15 signals and in accordance with the detected face information, 

a plurality of normalized image signals being obtained from 
the face pattern normalizing process, and 

iii) editing means for laying out a plurality of 
images, which are represented by the normalized image signals, 
20 in a predetermined layout, and obtaining the layout image signal 

representing the thus formed layout image. 

The present invention further provides a recording 
medium, on which a program for causing a computer to execute 
the image processing method in accordance with the present 
25 invention has been recorded and from which the computer is 
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capable of reading the program. 

With the image processing method and apparatus in 
accordance with the present invention , the face patterns of 
the persons in the person images are normalized, and the 
5 normalized image signals are thereby obtained. The images, 

which are represented by the normalized image signals, are laid 
out in the predetermined layout, and the layout image signal 
Q representing the thus formed layout image is obtained, 

=5: Therefore, in the layout image reproduced from the layout image 

10 signal, the positions and/or the sizes of the face patterns 

2 of the persons become uniform. Accordingly, the layout image, 

.^^ in which the positions and/or the sizes of the face patterns 

U. of the persons are uniform, can be formed efficiently. As a 

result , the productivity of layout images , such as school photos , 
^ 15 can be enhanced, 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is a block diagram showing an album printing 
system, in which an embodiment of the image processing apparatus 
in accordance with the present invention is employed, 
20 Figure 2 is an explanatory view showing a person 

image. 

Figure 3 is a flow chart showing how the embodiment 
of the image processing apparatus in accordance with the present 
invention operates, 
2 5 Figure 4 is a schematic view showing a layout image, 

6 



in which a plurality of person images have been normalized. 
Figure 5 is a schematic view showing a layout image, 
in which a plurality of person images have not been normalized, 
and 

Figure 6 is a schematic view showing layout images 
obtained with a different embodiment of the image processing 
apparatus in accordance with the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention will hereinbelow be described 
in further detail with reference to the accompanying drawings. 

Figure 1 is a block diagram showing an album printing 
system, in which an embodiment of the image processing apparatus 
in accordance with the present invention is employed. With 
reference to Figure 1, the album printing system comprises 
reading means 1 for reading out a plurality of original image 
signals SO, SO, each of which represents a person image. 

The album printing system also comprises detection means 2 for 
detecting face information H from each of the original image 
signals SO, SO, ... having been received from the reading means 
1. The face information H represents a displacement quantity, 
which is to be utilized in a face pattern normalizing process 
for uniformizing the positions and/or the sizes of face patterns 
in the person images. The album printing system further 
comprises normalization means 3 for performing affine 
transformation on each of the original image signals SO , SO , . . . 
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and in accordance with the corresponding face information H, 
and thereby obtaining a plurality of normalized image signals 
Sir SI, • • • representing images, in which the positions and/or 
the sizes of the face patterns of the persons have been 
5 normalized. The album printing system still further comprises 

editing means 4 for laying out the images, which are represented 
by the normalized image signals SI, SI, . . . , in a predetermined 
layout, and obtaining a layout image signal 32 representing 
a layout image R, in which the plurality of the person images 

10 have been laid out. The album printing system also comprises 

a printer 5 for reproducing the layout image R from the layout 
image signal S2 . 

The original image signals SO , SO , ... may be obtained 
from a read-out apparatus, such as a scanner, which reads out 

15 the person images from negative film or positive film. 

Alternatively, the original image signals SO, SO, ... may be 
obtained from image sensing operations performed by a digital 
camera. The thus obtained original image signals SO, SO, ... 
are stored on a storage medium or a storage device. The reading 

20 means 1 reads the original image signals SO, SO, ... from the 

storage medium or the storage device. 

The detection means 2 comprises edge detecting means 
6 for detecting edges in the face pattern embedded in the person 
image, which is represented by each original image signal SO. 

25 The detection means 2 also comprises binarization means 7 for 



binarizing the original image signal SO, which has been obtained 
from the edge detection, and obtaining a binary signal SO*. 
The detection means 2 further comprises pattern matching means 
8. The pattern matching means 8 performs a pattern matching 
process on the binary signal SO' and a reference signal G, which 
represents a predetermined binary reference image. The 
pattern matching means 8 thus detects at least two points, at 
which the degree of correlation between the person image and 
the reference image is highest. Also, the pattern matching 
means 8 determines displacement quantities, which are to be 
utilized for transforming a center position having coordinates 
(X, y) in the face pattern embedded in the person image as 
illustrated in Figure 2 into a center position having 
coordinates (xO, yO ) in the reference image, and uniformizing 
a size v of the face pattern in the vertical direction and a 
size h of the face pattern in the horizontal direction. The 
detection means 2 feeds information, which represents the thus 
determined displacement quantities, as the face information 
H into the normalization means 3 . The reference image 
represents a contour image of a face having a predetermined 
size, which contour image is located such that it may have the 
center position at the predetermined coordinates (xO, yO ) . 

In accordance with the face information H received 
from the detection means 2, the normalization means 3 performs 
affine transformation on the original image signal SO and 



obtains a normalized image signal SI* The affine 
transformation is performed with Formula (1) shown below* 




/ a b\ /xl 




(1) 



in which ( xl , y 1 ) represents the coordinate values of the person 
image before being transformed, and (x2, y2) represents the 
coordinate values of the person image after being transformed. 



Formula (1), all of the image size enlargement or reduction 
of the person image represented by the original image signal 
SO, the rotation of the entire person image, and the parallel 
translation of the person image are achieved simultaneously. 

The coefficients a, b, c, and d in Formula (1) are 
determined in the manner described below. Specifically, 
Formula ( 1 ) can be divided into Formulas ( 2 ) and ( 3 ) shown below. 



The two points in the person image, which are detected by the 
pattern matching process as the points having the highest degree 
of correlation with the reference image, may have coordinates 
(xll, yll) and coordinates {xl2, yl2). Also, the two points 
in the reference image, which correspond to the two points in 
the person image, may have coordinates (x21, y21) and 



With the coordinate transformation performed with 



x2=axl+byH-c 



(2) 



y2=-bxl+ayl+d 



(3) 
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coordinates (x22,y22). In such cases , Formulas (4), (5), (6), 
and (7) shown below obtain, 

x21=axll+byll+c (4) 
x22=axl2+byl2+c (5) 
y21=-bxll+ayll+d (6) 
y22="bxl2+ayl2+d (7) 
The four coefficients a, b, c, and d can be calculated by solving 
Formulas (4), (5), (6), and (7). 

How this embodiment operates will be described 
hereinbelow. Figure 3 is a flow chart showing how the 
embodiment of the image processing apparatus in accordance with 
the present invention operates • 

With reference to Figure 3, firstly, in a step SI, 
the reading means 1 reads the original image signals SO, SO, ... 
from the storage medium or the storage device. In a step S2, 
each of the original image signals SO, SO, ... is fed into the 
detection means 2. In a step S3, the edge detecting means 6 
of the detection means 2 performs a differentiation process, 
or the like, on the original image signal SO and detects edges 
in the person image represented by the original image signal 
SO . In a step S4 , the binarization means 7 performs a binarizing 
process on the original image signal SO, which has been obtained 
from the edge detection, the binary signal SO ' is obtained from 
the binarizing process. Thereafter, in a step S5, the pattern 
matching means 8 calculates the displacement quantities, which 



are to be utilized for transforming the center position having 
the coordinates (x, y ) in the face pattern embedded in the person 
image into the center position having the coordinates (xO, yO) 
in the reference image, and uniformizing the size v of the face 
pattern in the vertical direction and the size h of the face 
pattern in the horizontal direction. The information, which 
represents the calculated displacement quantities, is fed as 
the face information H into the normalization means 3. 

In a step S6, the normalization means 3 calculates 
the affine transformation coefficients with Formula (1) and 
in accordance with the face information H, performs the affine 
transformation on the original image signal SO, and obtains 
the normalized image signal Si representing the image, in which 
the position and/or the size of the face pattern of the person 
has been normalized. In a step 87, a judgment is made as to 
whether all of the original image signals SO, SO, ... having 
been read by the reading means 1 have been or have not been 
normalized. In cases where it has been judged that all of the 
original image signals SO, SO, ... have not been normalized, 
the processes from the step S2 to the step S6 are iterated. 
In cases where it has been judged that all of the original image 
signals SO, SO, ... have been normalized, in a step S8, the 
editing means 4 lays out the images, which are represented by 
the normalized image signals SI, SI, in a predetermined 

layout and obtains the layout image signal S2 representing the 



layout image R. Thereafter, in a step S9, the printer 5 prints 
the layout image R in accordance with the layout image signal 
32. 

In this manner, with this embodiment, the process 
for normalizing the face patterns of the persons in the person 
images is performed on the original image signals SO, SO, ... 
representing the person images, and the normalized image 
signals SI, SI, ... are thereby obtained. The images, which 
are represented by the normalized image signals SI, SI, 
are laid out in the predetermined layout, and the layout image 
signal S2 representing the thus formed layout image is obtained. 
Therefore, as illustrated in Figure 4, in the layout image R 
reproduced from the layout image signal S2, the positions and/or 
the sizes of the face patterns of the persons become uniform. 
Accordingly, the layout image R, in which the positions and/or 
the sizes of the face patterns of the persons are uniform, can 
be formed efficiently. As a result, the productivity of layout 
images R, R, such as school photos, can be enhanced. 

In the embodiment described above, the detection 
means 2 detects at least two points having the highest degree 
of correlation by performing the pattern matching process and 
calculates the affine transformation coefficients a, b, c, and 
d to be utilized for the normalization of the position and/or 
the size of the face pattern of the person in the person image. 
In cases where only the position of the face pattern of the 




person in the person image is to be compensated for, only one 
point having the highest correlation may be detected, and the 
position of the face pattern may be uniformized in accordance 
with the detected point. In this manner, the normalization of 
the face pattern can be performed quickly - 

Also, in the embodiment described above, the 
detection means 2 detects the edges in the person image, 
performs the pattern matching with the reference image, and 
thereby calculates the displacement quantities. 
Alternatively, the displacement quantities may be calculated 
with a technique described in, for example, Japanese Unexamined 
Patent Publication No. 6 ( 1994 ) -67320 . Specifically, the 
person image may be divided into a plurality of regions in 
accordance with the distributions of the hue and saturation 
values in the person image, and a face candidate region may 
be detected. A face region may then be detected in accordance 
with the shape of a neighboring region, which is located in 
the vicinity of the face candidate region. Thereafter, the 
pattern matching process as described above may be performed 
on the signal representing the face region, and the displacement 
quantities may thereby be calculated. As another alternative, 
an ellipse circumscribed about a simply detected face candidate 
region may be formed, and the displacement quantities with 
respect to the reference image may be calculated by taking the 
center point of the ellipse as the center position of the face 
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pattern having the coordinates (x, y) . The face region may be 
detected with one of various other techniques. For example, 
the face region may be detected by utilizing neural network 
techniques described in, for example, Japanese Unexamined 
Patent Publication Nos. 5 ( 1993 ) -274438 and 5 ( 1993 ) -307605 . 

Further, in the aforesaid embodiment, the layout 
image R is formed by normalizing the face patterns of a plurality 
of different persons. Alternatively, as illustrated in Figure 
6, an album may be formed by combining a plurality of layout 
images, each of which comprises a plurality of images of the 
same person. In such cases, the image processing apparatus in 
accordance with the present invention can be utilized for 
normalizing the positions and/or the sizes of the face patterns , 
which are embedded in the images constituting each layout image. 



